Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Efficient storage scheme for deadline aware distributed matrix multiplication
Yongzhu ZHAO, Weidong LI, Bin TANG, Feng MEI, Wenda LU
Journal of Computer Applications    2020, 40 (2): 311-315.   DOI: 10.11772/j.issn.1001-9081.2019091640
Abstract457)   HTML15)    PDF (742KB)(543)       Save

Distributed matrix multiplication is a fundamental operation in many distributed machine learning and scientific computing applications, but its performance is greatly influenced by the stragglers commonly existed in the systems. Recently, researchers have proposed a fountain code based coded matrix multiplication method, which can effectively mitigate the effect of stragglers by fully exploiting the partial results of stragglers. However, it lacks the consideration of the storage cost of worker nodes. By considering the tradeoff relationship between the storage cost and the finish time of computation, the computational deadline-aware storage optimization problem for heterogeneous worker nodes was proposed firstly. Then, through the theoretical analysis, the solution based on expectation approximation was presented, and the problem was transformed into a convex optimization problem by relaxation for efficient solution. Simulation results show that in the case of ensuring a large task success rate, the storage overhead of the proposed scheme will rapidly decrease as the task duration is relaxed, and the scheme can greatly reduce the storage overhead brought by encoding. In other words, the proposed scheme can significantly reduce the extra storage overhead while guaranteeing that the whole computation can be finished before the deadline with high probability.

Table and Figures | Reference | Related Articles | Metrics